The redshift where the first stars formed is an important and unknown milestone in cosmological structure formation. The evidence linking gamma ray bursts (GRBs) with star formation activity implies that the first GRBs occurred shortly after the first stars formed. Gamma ray bursts and their afterglows may thus offer a unique probe of this epoch, because they are bright from gamma ray to radio wavelengths and should be observable to very high redshift. Indeed, our ongoing near-IR followup programs already have the potential to detect bursts at redshift z ~ 10. In these proceedings, we discuss two distinct ways of using GRBs to probe the earliest star formation. First, direct GRB counts may be used as a proxy for star formation rate measurements. Second, high energy cutoffs in the GeV spectra of gamma ray bursts due to pair production with high redshift optical and ultraviolet background photons contain information on early star formation history. The second method is observationally more demanding, but also more rewarding, because each observed pair creation cutoff in a high redshift GRB spectrum will tell us about the integrated star formation history prior to the GRB redshift.